Effect of starch and protein on eating quality of japonica rice in Yangtze River Delta.

This study examined four types of japonica rice from Yangtze River Delta, categorized based on amylose content (AC) and protein content (PC): high AC with high PC, high AC with low PC, low AC with high PC, and low AC with low PC. It systematically explored the effect of starch, protein and their interactions on eating quality of japonica rice. Rheological analysis revealed that increased amylose, long chains amylopectin or protein levels during cooking strengthen starch-protein interactions (hydrogen bonding), forming a firm gel network. Scanning electron microscopy showed that increased amylose, long chains amylopectin or protein levels made protein and starch more stable in combination during cooking, limiting starch structure cleavage. Therefore, the eating quality of high AC in similar PC japonica rice and high PC in similar AC japonica rice were poor. Further, correlation and random-forest analysis (RFA) identified amylose as the most influential factor in starch-protein interactions affecting rice eating quality, followed by amylopectin and protein. RFA also revealed that in high AC japonica rice, the interactions of Fb3 and albumin with amylose were more conducive to forming good eating quality. In low AC japonica rice, the interactions of Fb2 and prolamin with amylose were more beneficial.

SipMaskv2: Enhanced Fast Image and Video Instance Segmentation.

We propose a fast single-stage method for both image and video instance segmentation, called SipMask, that preserves the instance spatial information by performing multiple sub-region mask predictions. The main module in our method is a light-weight spatial preservation (SP) module that generates a separate set of spatial coefficients for the sub-regions within a bounding-box, enabling a better delineation of spatially adjacent instances. To better correlate mask prediction with object detection, we further propose a mask alignment weighting loss and a feature alignment scheme. In addition, we identify two issues that impede the performance of single-stage instance segmentation and introduce two modules, including a sample selection scheme and an instance refinement module, to address these two issues. Experiments are performed on both image instance segmentation dataset MS COCO and video instance segmentation dataset YouTube-VIS. On MS COCO test-dev set, our method achieves a state-of-the-art performance. In terms of real-time capabilities, it outperforms YOLACT by a gain of 3.0% (mask AP) under the similar settings, while operating at a comparable speed. On YouTube-VIS validation set, our method also achieves promising results. The source code is available at https://github.com/JialeCao001/SipMask.

Visual-quality-driven unsupervised image dehazing.

Most of the existing learning-based dehazing methods require a diverse and large collection of paired hazy/clean images, which is intractable to obtain. Therefore, existing dehazing methods resort to training on synthetic images. This may result in a possible domain shift when treating real scenes. In this paper, we propose a novel unsupervised dehazing (lightweight) network without any reference images to directly predict clear images from the original hazy images, which consists of an interactive fusion module (IFM) and an iterative optimization module (IOM). Specifically, IFM interactively fuses multi-level features to make up for the missing information among deep and shallow features while IOM iteratively optimizes dehazed results to obtain pleasing visual effects. Particularly, based on the observation that hazy images usually suffer from quality degradation, four non-reference visual-quality-driven loss functions are designed to enable the network trained in an unsupervised way, including dark channel loss, contrast loss, saturation loss, and edge sharpness loss. Extensive experiments on two synthetic datasets and one real-world dataset demonstrate that our method performs favorably against the state-of-the-art unsupervised dehazing methods and even matches some supervised methods in terms of metrics such as PSNR, SSIM, and UQI.

Context and detail interaction network for stereo rain streak and raindrop removal.

Recently stereo image deraining has attracted lots of attention due to its superiority of abundant information from cross views. Exploring interaction information across stereo views is the key to improving the performance of stereo image deraining. In this paper, we design a general coarse-to-fine deraining framework for stereo rain streak and raindrop removal, called CDINet, comprising a stereo rain removal subnet and a stereo detail recovery subnet to restore images progressively. Two types of interaction modules are devised to explore interaction information for rain removal and detail recovery, respectively. Specifically, a global context interaction module is proposed to learn long-range dependencies of stereo images and remove rain by utilizing stereo structural information. A local detail interaction module is designed to model local contextual correlation, which aims at restoring the detail information by using neighborhood information from cross views. Extensive experiments are conducted on the two datasets including a synthetic rain streak removal dataset (RainKITTI) and a real raindrop removal dataset (Stereo Waterdrop), which demonstrates that our method sets new state-of-the-art deraining performance in terms of both quantitative and qualitative metrics with faster speed.

Hierarchical Regression and Classification for Accurate Object Detection.

Accurate object detection requires correct classification and high-quality localization. Currently, most of the single shot detectors (SSDs) conduct simultaneous classification and regression using a fully convolutional network. Despite high efficiency, this structure has some inappropriate designs for accurate object detection. The first one is the mismatch of bounding box classification, where the classification results of the default bounding boxes are improperly treated as the results of the regressed bounding boxes during the inference. The second one is that only one-time regression is not good enough for high-quality object localization. To solve the problem of classification mismatch, we propose a novel reg-offset-cls (ROC) module including three hierarchical steps: the regression of the default bounding box, the prediction of new feature sampling locations, and the classification of the regressed bounding box with more accurate features. For high-quality localization, we stack two ROC modules together. The input of the second ROC module is the output of the first ROC module. In addition, we inject a feature enhanced (FE) module between two stacked ROC modules to extract more contextual information. The experiments on three different datasets (i.e., MS COCO, PASCAL VOC, and UAVDT) are performed to demonstrate the effectiveness and superiority of our method. Without any bells or whistles, our proposed method outperforms state-of-the-art one-stage methods at a real-time speed. The source code is available at https://github.com/JialeCao001/HSD.

Non-Destructive Detection of Moldy Walnuts Based on Hyperspectral Imaging Technology.

Walnuts with their shells are a popular agricultural product in China. However, mildew from growth can sometimes be processed into foods. It is difficult to visually determine which walnuts have mildew without breaking the shells. A non-destructive method for detecting walnuts with mildew was studied by combining spectral data with image information. A total of 120 "Lüling" walnuts with shells were used for the mildew experiment. The characteristics of the spectral data from six surfaces of all samples were collected in the range of 370-1042 nm on days 0, 15, and 30. The spectrum was pretreated using SNV, and the feature bands were extracted using PCA and modeled using a support vector machine (SVM). The results show that the overall classification accuracy was 93%, with an of accuracy of 100% for INEN walnuts (normal internally and externally). The accuracy for IMEM walnuts (mildew internally and externally) reached 87.29%. There was an accuracy of 78.6% for IMEN walnuts (mildew internally and normal externally). The non-destructive detection of mildewed walnuts can be undertaken using hyperspectral imaging technology, which provides a new technique for exploring the mechanisms of walnuts with mildew.

Continuous Seizure Detection Based on Transformer and Long-Term iEEG.

Automatic seizure detection algorithms are necessary for patients with refractory epilepsy. Many excellent algorithms have achieved good results in seizure detection. Still, most of them are based on discontinuous intracranial electroencephalogram (iEEG) and ignore the impact of different channels on detection. This study aimed to evaluate the proposed algorithm using continuous, long-term iEEG to show its applicability in clinical routine. In this study, we introduced the ability of the transformer network to calculate the attention between the channels of input signals into seizure detection. We proposed an end-to-end model that included convolution and transformer layers. The model did not need feature engineering or format transformation of the original multi-channel time series. Through evaluation on two datasets, we demonstrated experimentally that the transformer layer could improve the performance of the seizure detection algorithm. For the SWEC-ETHZ iEEG dataset, we achieved 97.5% event-based sensitivity, 0.06/h FDR, and 13.7 s latency. For the TJU-HH iEEG dataset, we achieved 98.1% event-based sensitivity, 0.22/h FDR, and 9.9 s latency. In addition, statistics showed that the model allocated more attention to the channels close to the seizure onset zone within 20 s after the seizure onset, which improved the explainability of the model. This paper provides a new method to improve the performance and explainability of automatic seizure detection.

Non-linear perceptual multi-scale network for single image super-resolution.

Recently, deep convolutional neural networks (CNNs) have been widely explored in single image super-resolution (SISR) and achieved remarkable progress. However, most of the existing CNN-based SISR networks with a single-stream structure fail to make full use of the multi-scale features of low-resolution (LR) image. While those multi-scale SR models often integrate the information with different receptive fields by means of linear fusion, which leads to the redundant feature extraction and hinders the reconstruction performance of the network. To address both issues, in this paper, we propose a non-linear perceptual multi-scale network (NLPMSNet) to fuse the multi-scale image information in a non-linear manner. Specifically, a novel non-linear perceptual multi-scale module (NLPMSM) is developed to learn more discriminative multi-scale feature correlation by using high-order channel attention mechanism, so as to adaptively extract image features at different scales. Besides, we present a multi-cascade residual nested group (MC-RNG) structure, which uses a global multi-cascade mechanism to organize multiple local residual nested groups (LRNG) to capture sufficient non-local hierarchical context information for reconstructing high-frequency details. LRNG uses a local residual nesting mechanism to stack NLPMSMs, which aims to form a more effective residual learning mechanism and obtain more representative local features. Experimental results show that, compared with the state-of-the-art SISR methods, the proposed NLPMSNet performs well in both quantitative metrics and visual quality with a small number of parameters.

From Handcrafted to Deep Features for Pedestrian Detection: A Survey.

Pedestrian detection is an important but challenging problem in computer vision, especially in human-centric tasks. Over the past decade, significant improvement has been witnessed with the help of handcrafted features and deep features. Here we present a comprehensive survey on recent advances in pedestrian detection. First, we provide a detailed review of single-spectral pedestrian detection that includes handcrafted features based methods and deep features based approaches. For handcrafted features based methods, we present an extensive review of approaches and find that handcrafted features with large freedom degrees in shape and space have better performance. In the case of deep features based approaches, we split them into pure CNN based methods and those employing both handcrafted and CNN based features. We give the statistical analysis and tendency of these methods, where feature enhanced, part-aware, and post-processing methods have attracted main attention. In addition to single-spectral pedestrian detection, we also review multi-spectral pedestrian detection, which provides more robust features for illumination variance. Furthermore, we introduce some related datasets and evaluation metrics, and a deep experimental analysis. We conclude this survey by emphasizing open problems that need to be addressed and highlighting various future directions. Researchers can track an up-to-date list at https://github.com/JialeCao001/PedSurvey.

Phosphorous doped g-C3N4 supported cobalt phthalocyanine: An efficient photocatalyst for reduction of CO2 under visible-light irradiation.

The photoreduction of the green-house gas CO2 into carbon monoxide (CO) is a growing process due to the use of CO for the production of methanol in the Fischer-Tropsch process and the synthesis of many of the bulk chemicals. Here, we have synthesized phosphorous doped graphitic carbon nitride (P-g-C3N4) sensitized by the cobalt phthalocyanine complex for the molecular reduction of CO2 into CO under visible-light irradiation-the doping of phosphorous improved the stability as well as the harvesting of the visible region. The CoPc@P-g-C3N4 hybrid photocatalyst exhibited the highest efficiency for the photoreduction of CO2 with a high yield of 295 µmol-g-1 for CO under the experimental conditions. Also, hydrogen with low concentration was identified as a by-product under the experimental conditions. The photocatalyst had stability for six consecutive runs with negligible loss of the activity and no leaching of the cobalt content at the end of the sixth run of the photoreduction experiment. The stability of the photocatalysts is an advantage, which made it a suitable candidate for the current reaction system.

Improving Single Shot Object Detection With Feature Scale Unmixing.

Due to the advantages of real-time detection and improved performance, single-shot detectors have gained great attention recently. To solve the complex scale variations, single-shot detectors make scale-aware predictions based on multiple pyramid layers. Typically, small objects are detected on shallow layers while large objects are detected on deep layers. However, the features in the pyramid are not scale-aware enough, which limits the detection performance. Two common problems in single-shot detectors caused by object scale variations can be observed: (1) false negative problem, i.e., small objects are easily missed due to the weak features; (2) part-false positive problem, i.e., the salient part of a large object is sometimes detected as an object. With this observation, a new Neighbor Erasing and Transferring (NET) mechanism is proposed for feature scale-unmixing to explore scale-aware features in this paper. In NET, a Neighbor Erasing Module (NEM) is designed to erase the salient features of large objects and emphasize the features of small objects in shallow layers. A Neighbor Transferring Module (NTM) is introduced to transfer the erased features and highlight large objects in deep layers. With this mechanism, a single-shot network called NETNet is constructed for scale-aware object detection. In addition, we propose to aggregate nearest neighboring pyramid features to enhance our NET. Experiments on MS COCO dataset and UAVDT dataset demonstrate the effectiveness of our method. NETNet obtains 38.5% AP at a speed of 27 FPS and 32.0% AP at a speed of 55 FPS on MS COCO dataset. As a result, NETNet achieves a better trade-off for real-time and accurate object detection.

TJU-DHD: A Diverse High-Resolution Dataset for Object Detection.

Vehicles, pedestrians, and riders are the most important and interesting objects for the perception modules of self-driving vehicles and video surveillance. However, the state-of-the-art performance of detecting such important objects (esp. small objects) is far from satisfying the demand of practical systems. Large-scale, rich-diversity, and high-resolution datasets play an important role in developing better object detection methods to satisfy the demand. Existing public large-scale datasets such as MS COCO collected from websites do not focus on the specific scenarios. Moreover, the popular datasets (e.g., KITTI and Citypersons) collected from the specific scenarios are limited in the number of images and instances, the resolution, and the diversity. To attempt to solve the problem, we build a diverse high-resolution dataset (called TJU-DHD). The dataset contains 115354 high-resolution images (52% images have a resolution of 1624×1200 pixels and 48% images have a resolution of at least 2, 560×1.440 pixels) and 709 330 labeled objects in total with a large variance in scale and appearance. Meanwhile, the dataset has a rich diversity in season variance, illumination variance, and weather variance. In addition, a new diverse pedestrian dataset is further built. With the four different detectors (i.e., the one-stage RetinaNet, anchor-free FCOS, two-stage FPN, and Cascade R-CNN), experiments about object detection and pedestrian detection are conducted. We hope that the newly built dataset can help promote the research on object detection and pedestrian detection in these two scenes. The dataset is available at https://github.com/tjubiit/TJU-DHD.

Taking a Look at Small-Scale Pedestrians and Occluded Pedestrians.

Small-scale pedestrian detection and occluded pedestrian detection are two challenging tasks. However, most state-of-the-art methods merely handle one single task each time, thus giving rise to relatively poor performance when the two tasks, in practice, are required simultaneously. In this paper, it is found that small-scale pedestrian detection and occluded pedestrian detection actually have a common problem, i.e., an inaccurate location problem. Therefore, solving this problem enables to improve the performance of both tasks. To this end, we pay more attention to the predicted bounding box with worse location precision and extract more contextual information around objects, where two modules (i.e., location bootstrap and semantic transition) are proposed. The location bootstrap is used to reweight regression loss, where the loss of the predicted bounding box far from the corresponding ground-truth is upweighted and the loss of the predicted bounding box near the corresponding ground-truth is downweighted. Additionally, the semantic transition adds more contextual information and relieves semantic inconsistency of the skip-layer fusion. Since the location bootstrap is not used at the test stage and the semantic transition is lightweight, the proposed method does not add many extra computational costs during inference. Experiments on the challenging CityPersons and Caltech datasets show that the proposed method outperforms the state-of-the-art methods on the small-scale pedestrians and occluded pedestrians (e.g., 5.20% and 4.73% improvements on the Caltech).

Learning Multilayer Channel Features for Pedestrian Detection.

Pedestrian detection based on the combination of convolutional neural network (CNN) and traditional handcrafted features (i.e., HOG+LUV) has achieved great success. In general, HOG+LUV are used to generate the candidate proposals and then CNN classifies these proposals. Despite its success, there is still room for improvement. For example, CNN classifies these proposals by the fully connected layer features, while proposal scores and the features in the inner-layers of CNN are ignored. In this paper, we propose a unifying framework called multi-layer channel features (MCF) to overcome the drawback. It first integrates HOG+LUV with each layer of CNN into a multi-layer image channels. Based on the multi-layer image channels, a multi-stage cascade AdaBoost is then learned. The weak classifiers in each stage of the multi-stage cascade are learned from the image channels of corresponding layer. Experiments on Caltech data set, INRIA data set, ETH data set, TUD-Brussels data set, and KITTI data set are conducted. With more abundant features, an MCF achieves the state of the art on Caltech pedestrian data set (i.e., 10.40% miss rate). Using new and accurate annotations, an MCF achieves 7.98% miss rate. As many non-pedestrian detection windows can be quickly rejected by the first few stages, it accelerates detection speed by 1.43 times. By eliminating the highly overlapped detection windows with lower scores after the first stage, it is 4.07 times faster than negligible performance loss.

Cascade Learning by Optimally Partitioning.

Cascaded AdaBoost classifier is a well-known efficient object detection algorithm. The cascade structure has many parameters to be determined. Most of existing cascade learning algorithms are designed by assigning detection rate and false positive rate to each stage either dynamically or statically. Their objective functions are not directly related to minimum computation cost. These algorithms are not guaranteed to have optimal solution in the sense of minimizing computation cost. On the assumption that a strong classifier is given, in this paper, we propose an optimal cascade learning algorithm (iCascade) which iteratively partitions the strong classifiers into two parts until predefined number of stages are generated. iCascade searches the optimal partition point of each stage by directly minimizing the computation cost of the cascade. Theorems are provided to guarantee the existence of the unique optimal solution. Theorems are also given for the proposed efficient algorithm of searching optimal parameters . Once a new stage is added, the parameter for each stage decreases gradually as iteration proceeds, which we call decreasing phenomenon. Moreover, with the goal of minimizing computation cost, we develop an effective algorithm for setting the optimal threshold of each stage. In addition, we prove in theory why more new weak classifiers in the current stage are required compared to that of the previous stage. Experimental results on face detection and pedestrian detection demonstrate the effectiveness and efficiency of the proposed algorithm.

Learning Sampling Distributions for Efficient Object Detection.

Object detection is an important task in computer vision and machine intelligence systems. Multistage particle windows (MPW), proposed by Gualdi et al., is an algorithm of fast and accurate object detection. By sampling particle windows (PWs) from a proposal distribution (PD), MPW avoids exhaustively scanning the image. Despite its success, it is unknown how to determine the number of stages and the number of PWs in each stage. Moreover, it has to generate too many PWs in the initialization step and it unnecessarily regenerates too many PWs around object-like regions. In this paper, we attempt to solve the problems of MPW. An important fact we used is that there is a large probability for a randomly generated PW not to contain the object because the object is a sparse event relative to the huge number of candidate windows. Therefore, we design a PD so as to efficiently reject the huge number of nonobject windows. Specifically, we propose the concepts of rejection, acceptance, and ambiguity windows and regions. Then, the concepts are used to form and update a dented uniform distribution and a dented Gaussian distribution. This contrasts to MPW which utilizes only on region of support. The PD of MPW is acceptance-oriented whereas the PD of our method (called iPW) is rejection-oriented. Experimental results on human and face detection demonstrate the efficiency and the effectiveness of the iPW algorithm. The source code is publicly accessible.

Pedestrian Detection Inspired by Appearance Constancy and Shape Symmetry.

Most state-of-the-art methods in pedestrian detection are unable to achieve a good trade-off between accuracy and efficiency. For example, ACF has a fast speed but a relatively low detection rate, while checkerboards have a high detection rate but a slow speed. Inspired by some simple inherent attributes of pedestrians (i.e., appearance constancy and shape symmetry), we propose two new types of non-neighboring features: side-inner difference features (SIDF) and symmetrical similarity features (SSFs). SIDF can characterize the difference between the background and pedestrian and the difference between the pedestrian contour and its inner part. SSF can capture the symmetrical similarity of pedestrian shape. However, it is difficult for neighboring features to have such above characterization abilities. Finally, we propose to combine both non-neighboring features and neighboring features for pedestrian detection. It is found that non-neighboring features can further decrease the log-average miss rate by 4.44%. The relationship between our proposed method and some state-of-the-art methods is also given. Experimental results on INRIA, Caltech, and KITTI data sets demonstrate the effectiveness and efficiency of the proposed method. Compared with the state-of-the-art methods without using CNN, our method achieves the best detection performance on Caltech, outperforming the second best method (i.e., checkerboards) by 2.27%. Using the new annotations of Caltech, it can achieve 11.87% miss rate, which outperforms other methods.